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Abstract. In molecular pathology numerical chromosome aberrations have been found to be decisive for the prognosis of 
malignancy in tumours. The existence of such aberrations con be detected by interphase fluorescence .n situ hybndiMiion 
(FISH) The gain or loss of certain base sequences in the desoxy ribonucleic acid (DNA) can be estimated by counting the 
number or FISH signals per cell nucleus. The quanmaiive evaluation of such event* is a necessary condition for a prospect, ve 
use in diagnostic pathology. To avoid occlusions of signals, .he cell nucleus has to be analyzed in three C-Jg 
laser scanning microscopy is the means to obtain series of optical thin sections from fluorescence stained or marked material 
to fulfill the conditions mentioned above. inn 

A eraphical user interface (GUI) to a software package for display, inspection, count and (serai-)aulomauc analysis ot i-U 
images for pathologists is outlined including the underlying methods of 3-D image interaction and segmentation developed. 
The preparative methods are briefly described. Main emphasis is given to the methodical questions of computer-aided 
analysis of large 3-D image data sets for pathologists. Several automated analysis steps can be performed for segmentation 
and succeeding quantification. .„ . . . _ 

However tumour material is in contrast to isolated or cultured cells even for v,sual .nspecuon. a difficult matenal. For. 
the present a fully automated digital image analysis of 3-D data is not in sight. A semi-automatic segmentation melhod is 
thus presented here. 

Keywords: Fluorescence in situ hybridization (FISH), confocal laser scanning microscopy, graphical user interface. 3-D 
image analysis, segmentation 
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1. Introduction 

/./. Molecular pathology 

Prostate cancer is currently a leading cause of deaths from malignancies in American and European 
men [1 1 1. Owing lo improved methods of detection, the number of cases being reported has increased 
sharply. The clinical course of prostate cancer i.s often unpredictable, and various efforts have been 
undertaken to improve reliability of prognostic parameters. Morphological grading, staging, and 
tumour size arc mainly used to assess the malignant potential of prostate cancer [7], Besides these 
methods, new techniques like nucleic acid fluorescence in situ hybridization (FISH) have become 
increasingly important in diagnostic and research pathology. Using FISH numerical chromosome 
aberrations, e.g., gain or loss of specific single chromosomes (e.g., trisomy or monosomy) can be 
detected. Several studies have shown [2,8] that trisomy of chromosome 7 is found in malignant and 
non-malignant tumours of lung, kidney, and brain, as well as in tumours of the prostata. Considering 
that the biological significance of this alteration is still under discussion Zitzelsberger et al. [18] have 
shown that trisomy of chromosome 7 may be used as a prognostic factor because of the correlation 
with the development of lymph node metastases and grade uf differentiation. Therefore, an improved 
image analysis procedure for quantitative evaluation of FISH in sections of prostate cancer has been 
established to simplify its quantitative evaluation, to confirm its prognostic value as well as ro correlate 
FISH data with clinical follow-up. 

12. The need for 3-D FISH signal count 

A reliable count of FISH signals in tumour tissue necessitates a clear recognition of the tumour 
cells. This is only possible in sections with recognisable morphology. Standard histological sections 
contain only few cell nuclei completely inside the section. Only for those nuclei, a reliable count of 
FISH signals can be assured [1], hence thick sections have to be analysed for sufficient number of 
cells. 

However, such thick sections cannot be analysed in 2-D because of the degraded image quality and 
of the possible and frequent occlusion of FISH signals. The latter can be suspected from the relatively 
stable amount of monosomies mostly independent of the number of tri- and polysemies [16]. 

In addition, the analysis of 3-D images necessitates new training for the pathologists. Since the 
actual state of 3-D data representation (cyberspace, virtual reality) is far from routine applicability the 
interactive display and processing this approach has to be as near as possible to the typical observation 
behaviour of pathologists. 

1,3. Image acquisition, processing and analysis 

The seemingly simple task of signal counts becomes difficult under the condition thai the count 
has to be done with respect to a single cell nucleus. When these nuclei are isolated it is easy to 
estimate the membership of a signal to a nucleus. However, in most tumour tissues this relation is 
difficult or may be impossible to determine. This is true for visual inspection and even more so for 
computer based analysis. It is a well known fact that segmentation is the most cumbersome task in 
any digital image analysis approach. Recently another graphical user interface for visual inspection 
was presented [3] using primarily volume techniques for display. In our approach we tried to mimic 
the inspection behaviour of pathologists by mainly presenting sections, whatever volume rendering is 
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possihle. The interactive inspection part is already used for a comparison of thin and thick section 
FISH count already reported in [I J. 

Concerning segmentation of nuclei in 3-D in [4,15,17] proposed segmentation methods are mainly 
based on 2-D segmentation schemes, and more or less elaborated merging operations. As mentioned 
above, the segmentation of isolated nuclei is a minor problem. Therefore wc address preferentially 
the problem of connected nuclei by interactive means. 

2. Material: preparation, staining and sampling 

Tissue sections: 15 ;/m thick paraffin sections from routinely processed prostate cancer tissue were 
deparaffinized and rehydrated. 

DNA probes an/1 probe labeling: a -sate I Hie DNA probes specific for the ccntromeric region of 
» chromosome 7 were generated from clone 680TA [18]. The DNA probe was labeled by nick-translation 
with biotin-dUTP (Boehringer, Mannheim) according standard procedure [18]. 

Hybridization: The fluorescence in situ hybridization (FISH) was performed according to 1 1 J . Briefly, 
the slides were treated with pronase E (0.05% in 2xSSC, Merck, Darmstadt, Germany) at 37'C for 
20 to 35 min. After denaturation (70% formamide/2xSSC) at 72°C for 20 to 45 inin, the slides 
were dehydrated in an ethanol series. The hybridization mix consisted of carrier DNA (herring sperm 
DNA, SIGMA), 55% formamide/lxSSC, 10% dextran sulphate and the labeled DNA probe. This 
mixture was applied to the slides under a glass cover slip and sealed with rubber cement. After 
overnight incubation at 37 °C post-hybridization washes were performed at 43 °C in three changes of 
50% formamide/2xSSC PN buffer [9], and 3% bovine serum albumin (BSA, SIGMA). 

Detection and counter stain: The biotin-labelled centromere 7 probe was detected by streptavidin- 
fluorescein isothiocyanate (FITC) and biotinylated anti-streplavidin conjugates (Camon, Wiesbaden). 
The nuclei were counter stained with propidium iodide (PI, SIGMA) and mounted with an ami fade 
solution as described by [5]. Staining procedure was performed according to [14]. 

Sampling: Representative tumour and non-tumour control regions with unambiguous FISH signals 
and well-preserved nuclei morphology were selected for acquisition by CLSM. 

3. Methods 

The development and application of image analysis procedures are performed under the software 
package IDL (Interactive Data Language, Research Systems Inc., Boulder, CO, USA). As hardware 
development platform, a Silicon Graphics work station INDY, is used with 128 MB main memory, 
true color display, connected via local area network to other computers and PC's, especially the PC 
attached to the laser scanning microscope (LSM 410, Zeiss, Oberkochen, BRD). 

Beside mere memory problems with 3-D images of sizes about 15 MB pixels, even the processing 
time increases considerably. For example, even a RISC processor of MIPS 4400 type is slow for 
methods usually applied and well known in 2-D image analysis. 
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3.1. Image acquisition 

Fluorescence images are scanned using a Confocal Laser Scanning Microscope (CLSM) Zeiss LSM 
410. Essential setup features for the acquisition of FISH signals in tissue sections are as follows: Lens 
Zeiss PNF lOOx, 1.3, zoom = 2 ? realized by scanning unit). The scanned field of 62.5 x 62.5 //.in 
is sampled to 256 x 256 pixels giving a pixel size of 0.25 /im in x and y direction. Excitation laser 
hnes are selected according to the Huorochromes used. For propidium iodide (PI) used as a DNA 
counter stain and FITC labeled signals both are excited by the argon line 488 nni. The emission of 
both is measured simultaneously in two separate channels using a bandpass 515-565 nm for FITC 
and a longpass LP 590 nm for the PI channel, respectively. The axial distance selected between 
two subsequent confocal images depends on the further evaluation strategy. If a spatial isotropic 
representation of the 3-D data is intended, the axial distance is identical to the lateral distance of the 
pixels, i.e., 0.25 /<m. For a 16 jj.m thick section this results in a sequence of 64 scanned images. If 
only the sampling theorem should be mel. roughly a 0.5 j.in\ distance is enough, keeping in mind that 
the confocal axial resolution of a lens with NA = 1.3 is about the wavelength, i.e., 0.6 /xm [13]. The 
FITC emission is associated with the green channel and the PI emission with the red channel of a 
RGB true color image. The image data arc stored on disk in TIFF format. Scanning one field of view 
of a section with 16 /im thickness and using 0.25 /mi distance between individual images results in 
an uncompressed data set of 12.58 Mbyte. 

J. 2. Image display 

Various methods of displuy of digital volumes are proposed in this quickly growing field of me- 
thodical developments [3,6]. However, there arc still stringent requirements in research to solve the 
technical as well psychological problems of display of 3-D information. 

To allow an observation familiar to pathologists, a GUT has been designed (Fig. 1) displaying the 
3-D image as a heap of orthogonal sections with arbitrary choice of the cutting planes in x, y and 
z. Beside the counter stain volume in red colour channel and the signal in green, a second type of 
signal may be stored in blue colour channel. Possible display by interactions with the GUI are seen 
in Figs I and 2: 

1. Orthogonal sections of the original digitized volume with counter stain in red (R) and signal in 
green (G). 

2. Thresholded and segmented counter stain to improve recognition of nuclear borders with ad- 
justable parameters. 

3. Segmented signals with adjustable parameters. 

4. Histograms from all channels, counter stain (HISTJR), signal (HIST.G) and blue channel 
(HIST-B) (Fig. 1). 

5. Surface rendered counter stain images, (SLIC.R), signal (SLICG) and results of signal segmen- 
tation (SLICJB) (Fig. 3). 

6. Gallery of interactively selected box. sphere or ellipses around or semi-automatic segmented 
nuclei as a sequence of sections of sub-windows (Figs 4 and 5), 

7. Joumaling of the sequence of selected boxes, respectively, labelled objects including the signal 
count and other features. A replay function for repeated analysis of analysed locations is carried 
out. 
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Other possible interactions for analysis are histogram based local thresholding, cell labelling, FISH 
signal count, global and local segmentation, cutting of globally segmented touching cell nuclei by 
interactively adjustable planes (knife), spheres or ellipses, where the latter can be dragged around the 
object of interest and then used as limit for the volume growing process, as well as active volumes 
for segmentation, 3-D morphological operations, etc. 

3.3. Interactive volume processing 

For interactive volume analysis the orthogonal display of the volume can be positioned in x, y 
and z direction and a box, sphere or ellipses around the object of interest resized by dragging can be 
selected (Fig. 4). By another mou.se click the selected sub-volume is displayed in a separate window 
as a pallery of sub-sections (Fig. 5). On the left-hand side all sub-sections and on the right-hand side 
zoomed sub-sections are shown. ta 

During display the number of FISH signals are automatically displayed inside the selected sub- 
volume and calculated, and the mouse cursor jumps to the counting button's respective number 
(Fig. 2b). Clicking on the appropriate button stores the subvolume location, size and count. In 
the display field of orthogonal sections the box location is outlined for information of the already 
scanned places. The buttons SAVE, LOAD, CLEAR, REDRAW, DRAW Count and PRINT Count al- 
low storage and display of the count list (Fig. 2b). Also old count list files can be read and redisplayed 
for further inspection (Fig. 6). 

3.4. Semi-automatical volume processing 

3.4.1. Segmentation of nuclei 

The completely automatic segmentation of cell nuclei is still a distant dream. Wc have developed 
a seeded volume growing technique based on several size and shape constraints to segment the cell 
nuclei and to automatically count the FISH signal per nuclei. After a slight volume opening for noise 
reduction the image is subjected to global segmentation. It is automatically thresholded on the basis 
of local histograms and the 3-D connected components of the resulting two-level image are labelled. 
Cells which are out of focus and too complicated to segment are rejected from further evaluation. 
Also the cells which are at the border of the image are rejected since the completeness of such cell 
nuclei cannot be ascertained. Cells which are touching each other are first selected for semi-automatic 
segmentation. The center of such a cell is selected by clicking the mouse over the approximate 
centroid. This point will act as a seed and the seed is grown in all directions using the threshold 
derived from local histograms, 

The overshoot of the marked region into neighbouring cells which touch each other is checked by 
the global shape constraints like maximum extension and in a second step by an adjustable ellipsoid 
surface which is fitted in such a way that the cell nucleus falls just within. 

After separation of the touching cell nucleus, the image volume is subjected to a relabelling process 
to enable the newly segmented cells to have a distinct label (Fig. 7). Segmented cell regions are 
selected in the signal channel for further processing. 

3.4.2. Segmentation of FISH signals 

Several different methods, from thresholding via linear transformations to non-linear ones, e.g., 
transformations derived from mathematical morphology [10,12], were applied. For the actual material 
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with a small extension in :c, y and z direction FISH signals in 3-D arc represented by small grain-like 
structures. 

A top-hat transformation was considered (o be the best method to enhance the FISH signals for 
segmentation. The top-hat transformation image is the difference between an original grey scale image 
and an opened version. The radius of opening denotes the diameter of the cop-hat. A top-hat threshold 
can be considered as the height of the top-hat. Incidently, the term Lop-hat is derived from 2-D image 
analysis where the operation can be considered as putting a top-hat on the intensity profile. This image 
cannot easily be transferred to 3-D situations, where the signal intensity has to be considered as a 
fourth dimension. The applied top-hat transformation corresponds to the simplified algorithm proposed 
by [16J for 2-D compact spots, where additionally, a watershed transformation is necessary to detect 
small neighboring spots. The latter is nor necessary by applying the full 3-0 top-hat transformation. 
All operations are performed as 3-D operations. 

After thresholding the top-hat transformation, the resulting (binary). volume is processed by the 
selection of one connected object after ihe other, using again 3-D volume -growing. A grain is 
accepted as FISH signal after a size and shape criterion and its presence located inside the labelled 
nucleus. 

3.5. Featuring 

The goal of quantitative FISH to analyse about pathological alterations is the count of gains and 
losses of chromosomes. The physiologically detectable disomic nuclei (2 signals) in nunour tissue 
are not of primary interest. Of most importance is the number of appearing trisomies (3 signals), 
polysomies (> 3 signals) and monosomies (1 signal). The subjective computer-aided evaluation should 
equip us with data on the spatial distribution of signals in the nuclei as well as about the size and 
shape of nuclei in 3-D. 

3.5./. Interactive 

During visual inspection of the volume the coordinates and size of each selected box with the 
subjectively estimated signals are stored accompanied by the FISH signal count. This count list is 
subject to further analysis either by pooling count lists of several volumes or by using the box locations 
as expert selected seed for semi-automatical analysis, 

3.5.2. Semi -automatic 

First a global (threshold) based segmentation and object labelling is performed. The pathologist 
can display the result and decide which nuclei have to be counted and which are to be deleted. 
Selection or deletion are made by clicking on the labelled object. Improvement of segmentation is 
performed by selecting the object and proceeding as explained earlier under the section segmentation 
of nuclei. Forjabelled objects the FISH signals can be automatically counted, the box coordinates of 
the surrounding box, the count, signal and nucleus volume and the label number are stored; THfe FISH 
signals of either all labelled nuclei or only of interactively selected ones are counted. In the latter 
case a gallery * of aJbxxx determined by the size of the selected object can be displayed ^comparable. to._ 
interactive processing. 

Beside FISH signal count, the nuclei as well as each signal arc accessible for further feature extrac- 
tion such as calculation of volumes, mean total intensities and variations [4] and signal distribution in 
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4. Preliminary results and conclusions 



The interactive evaluation aid described above as well as the semi-automated evaluation, supp 
us with data for a more straightforward method for detecting suspected signals or ones that need 
be analyzed. This would allow to concentrate the analysis steps on small regions of interest in t 
volume data. 

Until now 19 tumours with about 300 image (volume) blocks have been visually evaluated usii 
this program. The results presented in [1] justify the increased workload by 3-D image analys 
FISH analysis in semi-thin sections deliver a serious underestimation compared to the signal numb 
in thick sections. The comparison of visually counted signals and semi-automatic counting in 3- 
will be communicated in the near future. 

The methodical proceedings for quantitative fluorescent in situ hybridization for 3-D evaluation h 
been outlined here and a program which can run on multiple platforms has been developed. Tl 
software is user friendly. The pathologists who are not familiar with computer aided image analyst 
can easily work with this program. The software is not just for FISH evaluation but can be used f 
morphological feature evaluation and to work on any kind of visualizable signal in the cell nucleus - 
any other object. The GUI can also be used to visualize the cell distribution and/or invasive patte 
of malignant cells in three dimensions. 
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